Recursive self-tuning control of finite Markov chains

Recursive Self-tuning Control of Finite Markov Chains

A recursive self-tuning control scheme for finite Markov chains is proposed wherein the unknown parameter is estimated by a stochastic approximation scheme for maximizing the log-likelihood function and the control is obtained via a relative value iteration algorithm. The analysis uses the asymptotic o.d.e.s associated with these.

Self - Learning Control of Finite Markov Chains

This book presents a collection of work on algorithms for learning in Markov decision processes. The problem addressed is very similar in spirit to “the reinforcement learning problem,” which has become a central topic in artificial intelligence (see, e.g., [8]). However, the book reflects an entirely different thread of research, and there has been surprisingly little interaction between this ...

Recursive filters for partially observable finite Markov chains

In this note, we consider discrete-time finite Markov Chains and assume that they are only partly observed. We obtain finite-dimensional normalized filters for basic statistics associated with such processes. Recursive equations for these filters are derived by means of simple computations involving conditional expectations. An application to the estimation of parameters for the so-called discr...

Finite Markov Chains

This paper provides basic information and theorems about finite Markov chains. The inspiration for this paper came from Professor Laci Babai’s Discrete Mathematics lecture notes from the REU program of 2003. After reading through his chapter on Markov chains, I decided to proceed by answering as many exercises from the notes as possible. Below is what I have finished.

Lectures on finite Markov chains